Propensity score matching and variations on the balancing test

Balancing tests are diagnostics designed for use with propensity score methods, a widely used non-experimental approach in the evaluation literature. Such tests provide useful information on whether plausible counterfactuals have been created. Currently, multiple balancing tests exist in the literature but it is unclear which is the most useful. This article highlights the poor size properties of commonly employed balancing tests and attempts to shed some light on the link between the results of balancing tests and bias of the evaluation estimator. The simulation results suggest that in scenarios where the conditional independence assumption holds, a permutation version of the balancing test described in Dehejia and Wahba (Rev Econ Stat 84:151–161, 2002) can be useful in applied study. The proposed test has good size properties. In addition, the test appears to have good power for detecting a misspecification in the link function and some power for detecting an omission of relevant non-linear terms involving variables that are included at a lower order.

Prognostic Relevance of Palliative Primary Tumor Removal in 37,793 Metastatic Colorectal Cancer Patients: A Population-Based, Propensity Score-Adjusted Trend Analysis

OBJECTIVE To assess whether palliative primary tumor resection in colorectal cancer patients with incurable stage IV disease is associated with improved survival. BACKGROUND There is a heated debate regarding whether or not an asymptomatic primary tumor should be removed in patients with incurable stage IV colorectal disease. METHODS Stage IV colorectal cancer patients were identified in the Surveillance, Epidemiology, and End Results database between 1998 and 2009. Patients undergoing surgery to metastatic sites were excluded. Overall survival and cancer-specific survival were compared between patients with and without palliative primary tumor resection using risk-adjusted Cox proportional hazard regression models and stratified propensity score methods. RESULTS Overall, 37,793 stage IV colorectal cancer patients were identified. Of those, 23,004 (60.9%) underwent palliative primary tumor resection. The rate of patients undergoing palliative primary cancer resection decreased from 68.4% in 1998 to 50.7% in 2009 (P < 0.001). In Cox regression analysis after propensity score matching primary cancer resection was associated with a significantly improved overall survival [hazard ratio (HR) of death = 0.40, 95% confidence interval (CI) = 0.39-0.42, P < 0.001] and cancer-specific survival (HR of death = 0.39, 95% CI = 0.38-0.40, P < 0.001). The benefit of palliative primary cancer resection persisted during the time period 1998 to 2009 with HRs equal to or less than 0.47 for both overall and cancer-specific survival. CONCLUSIONS On the basis of this population-based cohort of stage IV colorectal cancer patients, palliative primary tumor resection was associated with improved overall and cancer-specific survival. Therefore, the dogma that an asymptomatic primary tumor never should be resected in patients with unresectable colorectal cancer metastases must be questioned.

A comparison between accelerated hypofractionation and stereotactic ablative radiotherapy (SABR) for early-stage non-small cell lung cancer (NSCLC): Results of a propensity score-matched analysis

BACKGROUND AND PURPOSE: Stereotactic ablative radiotherapy (SABR) has become standard for inoperable early-stage non-small cell lung cancer (NSCLC). However, there is no randomized evidence demonstrating benefit over more fractionated radiotherapy. We compared accelerated hypofractionation (AH) and SABR using a propensity score-matched analysis.

MATERIALS AND METHODS: From 1997-2007, 119 patients (T1-3N0M0 NSCLC) were treated with AH (48-60Gy, 12-15 fractions). Prior to SABR, this represented our institutional standard. From 2008-2012, 192 patients (T1-3N0M0 NSCLC) were treated with SABR (48-52Gy, 4-5 fractions). A total of 114 patients (57 per cohort) were matched (1:1 ratio, caliper: 0.10) using propensity scores.

RESULTS: Median follow-up (range) for the AH cohort was 36.3 (2.5-109.1) months, while that for the SABR group was 32.5 (0.3-62.6)months. Three-year overall survival (OS) and local control (LC) rates were 49.5% vs. 72.4% [p=0.024; hazard ratio (HR): 2.33 (1.28, 4.23), p=0.006] and 71.9% vs. 89.3% [p=0.077; HR: 5.56 (1.53, 20.2), p=0.009], respectively. On multivariable analysis, tumour diameter and PET staging were predictive for OS, while the only predictive factor for LC was treatment cohort.

CONCLUSIONS: OS and LC were improved with SABR, although OS is more closely related to non-treatment factors. This represents one of the few studies comparing AH to SABR for early-stage lung cancer.

Statistiques appliquées en chirurgie cardiaque adulte : analyses de survie et applications du “propensity score”

L'objectif principal de ce travail est d’étudier en profondeur certaines techniques biostatistiques avancées en recherche évaluative en chirurgie cardiaque adulte. Les études ont été conçues pour intégrer les concepts d'analyse de survie, analyse de régression avec “propensity score”, et analyse de coûts. Le premier manuscrit évalue la survie après la réparation chirurgicale de la dissection aigüe de l’aorte ascendante. Les analyses statistiques utilisées comprennent : analyses de survie avec régression paramétrique des phases de risque et d'autres méthodes paramétriques (exponentielle, Weibull), semi-paramétriques (Cox) ou non-paramétriques (Kaplan-Meier) ; survie comparée à une cohorte appariée pour l’âge, le sexe et la race utilisant des tables de statistiques de survie gouvernementales ; modèles de régression avec “bootstrapping” et “multinomial logit model”. L'étude a démontrée que la survie s'est améliorée sur 25 ans en lien avec des changements dans les techniques chirurgicales et d’imagerie diagnostique. Le second manuscrit est axé sur les résultats des pontages coronariens isolés chez des patients ayant des antécédents d'intervention coronarienne percutanée. Les analyses statistiques utilisées comprennent : modèles de régression avec “propensity score” ; algorithme complexe d'appariement (1:3) ; analyses statistiques appropriées pour les groupes appariés (différences standardisées, “generalized estimating equations”, modèle de Cox stratifié). L'étude a démontrée que l’intervention coronarienne percutanée subie 14 jours ou plus avant la chirurgie de pontages coronariens n'est pas associée à des résultats négatifs à court ou long terme. Le troisième manuscrit évalue les conséquences financières et les changements démographiques survenant pour un centre hospitalier universitaire suite à la mise en place d'un programme de chirurgie cardiaque satellite. Les analyses statistiques utilisées comprennent : modèles de régression multivariée “two-way” ANOVA (logistique, linéaire ou ordinale) ; “propensity score” ; analyses de coûts avec modèles paramétriques Log-Normal. Des modèles d’analyse de « survie » ont également été explorés, utilisant les «coûts» au lieu du « temps » comme variable dépendante, et ont menés à des conclusions similaires. L'étude a démontrée que, après la mise en place du programme satellite, moins de patients de faible complexité étaient référés de la région du programme satellite au centre hospitalier universitaire, avec une augmentation de la charge de travail infirmier et des coûts.

Perceived fairness of pay among people with and without disabilities: a propensity score matched analysis of working Australians

OBJECTIVES: Equity and fairness at work are associated with a range of organizational and health outcomes. Past research suggests that workers with disabilities experience inequity in the workplace. It is difficult to conclude whether the presence of disability is the reason for perceived unfair treatment due to the possible confounding of effect estimates by other demographic or socioeconomic factors. METHODS: The data source was the Household, Income, and Labor Dynamics in Australia (HILDA) survey (2001-2012). Propensity for disability was calculated from logistic models including gender, age, education, country of birth, and father's occupational skill level as predictors. We then used nearest neighbor (on propensity score) matched analysis to match workers with disabilities to workers without disability. RESULTS: Results suggest that disability is independently associated with lower fairness of pay after controlling for confounding factors in the propensity score matched analysis; although results do suggest less than half a standard deviation difference, indicating small effects. Similar results were apparent in standard multivariable regression models and alternative propensity score analyses (stratification, covariate adjustment using the propensity score, and inverse probability of treatment weighting). CONCLUSIONS: Whilst neither multivariable regression nor propensity scores adjust for unmeasured confounding, and there remains the potential for other biases, similar results for the two methodological approaches to confounder adjustment provide some confidence of an independent association of disability with perceived unfairness of pay. Based on this, we suggest that the disparity in the perceived fairness of pay between people with and without disabilities may be explained by worse treatment of people with disabilities in the workplace.

Propensity score matching approach to test the association of income inequality and mortality in Sao Paulo, Brazil

Background Support for the adverse effect of high income inequality on population health has come from studies that focus on larger areas, such as the US states, while studies at smaller geographical areas (eg, neighbourhoods) have found mixed results. Methods We used propensity score matching to examine the relationship between income inequality and mortality rates across 96 neighbourhoods (distritos) of the municipality of Sao Paulo, Brazil. Results Prior to matching, higher income inequality distritos (Gini >= 0.25) had slightly lower overall mortality rates (2.23 per 10 000, 95% CI -23.92 to 19.46) compared to lower income inequality areas (Gini <0.25). After propensity score matching, higher inequality was associated with a statistically significant higher mortality rate (41.58 per 10 000, 95% CI 8.85 to 73.3). Conclusion In Sao Paulo, the more egalitarian communities are among some of the poorest, with the worst health profiles. Propensity score matching was used to avoid inappropriate comparisons between the health status of unequal (but wealthy) neighbourhoods versus equal (but poor) neighbourhoods. Our methods suggest that, with proper accounting of heterogeneity between areas, income inequality is associated with worse population health in Sao Paulo.

Assessment of candidemia-attributable mortality in critically ill patients using propensity score matching analysis

[EN] Introduction: Candidemia in critically ill patients is usually a severe and life-threatening condition with a high crude mortality. Very few studies have focused on the impact of candidemia on ICU patient outcome and attributable mortality still remains controversial. This study was carried out to determine the attributable mortality of ICU-acquired candidemia in critically ill patients using propensity score matching analysis. Methods: A prospective observational study was conducted of all consecutive non-neutropenic adult patients admitted for at least seven days to 36 ICUs in Spain, France, and Argentina between April 2006 and June 2007. The probability of developing candidemia was estimated using a multivariate logistic regression model. Each patient with ICU-acquired candidemia was matched with two control patients with the nearest available Mahalanobis metric matching within the calipers defined by the propensity score. Standardized differences tests (SDT) for each variable before and after matching were calculated. Attributable mortality was determined by a modified Poisson regression model adjusted by those variables that still presented certain misalignments defined as a SDT > 10%. Results: Thirty-eight candidemias were diagnosed in 1,107 patients (34.3 episodes/1,000 ICU patients). Patients with and without candidemia had an ICU crude mortality of 52.6% versus 20.6% (P < 0.001) and a crude hospital mortality of 55.3% versus 29.6% (P = 0.01), respectively. In the propensity matched analysis, the corresponding figures were 51.4% versus 37.1% (P = 0.222) and 54.3% versus 50% (P = 0.680). After controlling residual confusion by the Poisson regression model, the relative risk (RR) of ICU- and hospital-attributable mortality from candidemia was RR 1.298 (95% confidence interval (CI) 0.88 to 1.98) and RR 1.096 (95% CI 0.68 to 1.69), respectively. Conclusions: ICU-acquired candidemia in critically ill patients is not associated with an increase in either ICU or hospital mortality.

An overview of the objectives of and the approaches to propensity score analyses

The assessment of treatment effects from observational studies may be biased with patients not randomly allocated to the experimental or control group. One way to overcome this conceptual shortcoming in the design of such studies is the use of propensity scores to adjust for differences of the characteristics between patients treated with experimental and control interventions. The propensity score is defined as the probability that a patient received the experimental intervention conditional on pre-treatment characteristics at baseline. Here, we review how propensity scores are estimated and how they can help in adjusting the treatment effect for baseline imbalances. We further discuss how to evaluate adequate overlap of baseline characteristics between patient groups, provide guidelines for variable selection and model building in modelling the propensity score, and review different methods of propensity score adjustments. We conclude that propensity analyses may help in evaluating the comparability of patients in observational studies, and may account for more potential confounding factors than conventional covariate adjustment approaches. However, bias due to unmeasured confounding cannot be corrected for.

Statins are associated with decreased mortality in abdominal, but not in thoracic aortic aneurysm patients undergoing endovascular repair: propensity score-adjusted analysis

BACKGROUND: Purpose of this study was to compare the correlation of statin use with long-term mortality in patients with abdominal (AAA) and thoracic aortic aneurysm (TAA). PATIENTS AND METHODS: We compared long-term survival of 731 AAA and 59 TAA patients undergoing elective endovascular repair (EVAR). Kaplan-Meier survival curves were compared by the log-rank method. Propensity score-adjusted multivariable logistic regression models were used to determine independent associations of statin use on vital status after EVAR. RESULTS: Statin use was associated with decreased long-term mortality in AAA patients in bivariate and multivariable regression analysis, in which the effect of propensity to receive a statin was considered (adjusted HR: .613, 95%-CI: .379- .993, p = .047) whereas mortality of TAA patients was not associated with use of statins (adjusted HR: 1.795, 95%-CI: .147 -21.942, p = .647). CONCLUSIONS: Use of statins is an independent predictor of decreased mortality after elective EVAR in AAA, but not in TAA patients. These findings indirectly support the concept of a distinct pathogenesis of AAA and TAA.

A propensity score-matched comparison between Cardia and Amplatzer PFO closure devices - insights from the SOLUTION registry (Swiss percutaneOus patent foramen ovale cLosUre in recurrent clinical events prevenTION)

Aims: To compare clinical outcome of Amplatzer PFO (APFO) to Cardia PFO (CPFO) occluder. Percutaneous patent foramen ovale (PFO) closure prevents stroke recurrence in stroke due to paradoxical embolism. Methods and results: The primary endpoint was a composite of stroke, TIA, or peripheral embolism at follow-up. The secondary endpoint was residual shunt. Outcome was compared among 934 (APFO: 712; CPFO: 222) patients, and in 297 propensity score-matched patients. The primary endpoint occurred in 29 patients (0.71/100 patient-years): four (2%) with the CPFO (0.31/100 patient-years), and 25 (4%) with the APFO (0.89/100 patient-years) (p=0.20). Residual shunt at six months was more frequent with the CPFO (31% versus 9%, p<0.001). No differences in residual shunts were seen at the last available echocardiographic follow-up (9±18 months): APFO 11%, CPFO 14%, p=0.22. Conclusions: This study suggests that PFO closure with APFO or CPFO is equally effective for the prevention of recurrent events. Residual shunt was more frequent at six months with CPFO, but was similar to APFO at later follow-up.

Drug-eluting stents vs. bare metal stents in patients with cardiogenic shock: a comparison by propensity score analysis.

BACKGROUND In patients with cardiogenic shock, data on the comparative safety and efficacy of drug-eluting stents (DESs) vs. bare metal stents (BMSs) are lacking. We sought to assess the performance of DESs compared with BMSs among patients with cardiogenic shock undergoing percutaneous coronary intervention (PCI). METHODS Out of 236 patients with acute coronary syndromes complicated by cardiogenic shock, 203 were included in the final analysis. The primary endpoint included death, and the secondary endpoint of major adverse cardiac and cerebrovascular events (MACCEs) included the composite of death, myocardial infarction, any repeat revascularization and stroke. Patients were followed for a minimum of 30 days and up to 4 years. As stent assignment was not random, we performed a propensity score analysis to minimize potential bias. RESULTS Among patients treated with DESs, there was a lower risk of the primary and secondary endpoints compared with BMSs at 30 days (29 vs. 56%, P < 0.001; 34 vs. 58%, P = 0.001, respectively) and during long-term follow-up [hazard ratio 0.43, 95% confidence interval (CI) 0.29-0.65, P < 0.001; hazard ratio 0.49, 95% CI 0.34-0.71, P < 0.001, respectively]. After propensity score adjustment, all-cause mortality was reduced among patients treated with DESs compared with BMSs both at 30 days [adjusted odds ratio (OR) 0.26, 95% CI 0.11-0.62; P = 0.002] and during long-term follow-up (adjusted hazard ratio 0.40, 95% CI 0.22-0.72; P = 0.002). The rate of MACCE was lower among patients treated with DESs compared with those treated with BMSs at 30 days (adjusted OR 0.42, 95% CI 0.19-0.95; P = 0.036). The difference in MACCEs between devices approached significance during long-term follow-up (adjusted hazard ratio 0.60, 95% CI 0.34-1.01; P = 0.052). CONCLUSION DESs appear to be associated with improved clinical outcomes, including a reduction in all-cause mortality compared with BMSs among patients undergoing PCI for cardiogenic shock, possibly because of a pacification of the infarct-related artery by anti-inflammatory drug. The results of this observational study require confirmation in an appropriately powered randomized trial.

Comparison of prasugrel and clopidogrel-treated patients with acute coronary syndrome undergoing percutaneous coronary intervention: A propensity score-matched analysis of the Acute Myocardial Infarction in Switzerland (AMIS)-Plus Registry

OBJECTIVE The purpose of this study was to investigate outcomes of patients treated with prasugrel or clopidogrel after percutaneous coronary intervention (PCI) in a nationwide acute coronary syndrome (ACS) registry. BACKGROUND Prasugrel was found to be superior to clopidogrel in a randomized trial of ACS patients undergoing PCI. However, little is known about its efficacy in everyday practice. METHODS All ACS patients enrolled in the Acute Myocardial Infarction in Switzerland (AMIS)-Plus registry undergoing PCI and being treated with a thienopyridine P2Y12 inhibitor between January 2010-December 2013 were included in this analysis. Patients were stratified according to treatment with prasugrel or clopidogrel and outcomes were compared using propensity score matching. The primary endpoint was a composite of death, recurrent infarction and stroke at hospital discharge. RESULTS Out of 7621 patients, 2891 received prasugrel (38%) and 4730 received clopidogrel (62%). Independent predictors of in-hospital mortality were age, Killip class >2, STEMI, Charlson comorbidity index >1, and resuscitation prior to admission. After propensity score matching (2301 patients per group), the primary endpoint was significantly lower in prasugrel-treated patients (3.0% vs 4.3%; p=0.022) while bleeding events were more frequent (4.1% vs 3.0%; p=0.048). In-hospital mortality was significantly reduced (1.8% vs 3.1%; p=0.004), but no significant differences were observed in rates of recurrent infarction (0.8% vs 0.7%; p=1.00) or stroke (0.5% vs 0.6%; p=0.85). In a predefined subset of matched patients with one-year follow-up (n=1226), mortality between discharge and one year was not significantly reduced in prasugrel-treated patients (1.3% vs 1.9%, p=0.38). CONCLUSIONS In everyday practice in Switzerland, prasugrel is predominantly used in younger patients with STEMI undergoing primary PCI. A propensity score-matched analysis suggests a mortality benefit from prasugrel compared with clopidogrel in these patients.

Patient outcomes after laminotomy, hemilaminectomy, laminectomy and laminectomy with instrumented fusion for spinal canal stenosis: a propensity score-based study from the Spine Tango registry.

PURPOSE To compare patient outcomes and complication rates after different decompression techniques or instrumented fusion (IF) in lumbar spinal stenosis (LSS). METHODS The multicentre study was based on Spine Tango data. Inclusion criteria were LSS with a posterior decompression and pre- and postoperative COMI assessment between 3 and 24 months. 1,176 cases were assigned to four groups: (1) laminotomy (n = 642), (2) hemilaminectomy (n = 196), (3) laminectomy (n = 230) and (4) laminectomy combined with an IF (n = 108). Clinical outcomes were achievement of minimum relevant change in COMI back and leg pain and COMI score (2.2 points), surgical and general complications, measures taken due to complications, and reintervention on the index level based on patient information. The inverse propensity score weighting method was used for adjustment. RESULTS Laminotomy, hemilaminectomy and laminectomy were significantly less beneficial than laminectomy in combination with IF regarding leg pain (ORs with 95% CI 0.52, 0.34-0.81; 0.25, 0.15-0.41; 0.44, 0.27-0.72, respectively) and COMI score improvement (ORs with 95% CI 0.51, 0.33-0.81; 0.30, 0.18-0.51; 0.48, 0.29-0.79, respectively). However, the sole decompressions caused significantly fewer surgical (ORs with 95% CI 0.42, 0.26-0.69; 0.33, 0.17-0.63; 0.39, 0.21-0.71, respectively) and general complications (ORs with 95% CI 0.11, 0.04-0.29; 0.03, 0.003-0.41; 0.25, 0.09-0.71, respectively) than laminectomy in combination with IF. Accordingly, the likelihood of required measures was also significantly lower after laminotomy (OR 0.28, 95% CI 0.17-0.46), hemilaminectomy (OR 0.28, 95% CI 0.15-0.53) and after laminectomy (OR 0.39, 95% CI 0.22-0.68) in comparison with laminectomy with IF. The likelihood of a reintervention was not significantly different between the treatment groups. DISCUSSION As already demonstrated in the literature, decompression in patients with LSS is a very effective treatment. Despite better patient outcomes after laminectomy in combination with IF, caution is advised due to higher rates of surgical and general complications and consequent required measures. Based on the current study, laminotomy or laminectomy, rather than hemilaminectomy, is recommendable for minimum relevant pain relief.

Las becas y ayudas al estudio como elemento determinante de la continuidad escolar en el nivel secundario post-obligatorio : un análisis de sensibilidad a partir de la aplicación del Propensity Score Matching.

Resumen tomado de la publicación

Automated peritoneal dialysis is associated with better survival rates compared to continuous ambulatory peritoneal dialysis: a propensity score matching analysis

The impact of peritoneal dialysis modality on patient survival and peritonitis rates is not fully understood, and no large-scale randomized clinical trial (RCT) is available. In the absence of a RCT, the use of an advanced matching procedure to reduce selection bias in large cohort studies may be the best approach. The aim of this study is to compare automated peritoneal dialysis (APD) and continuous ambulatory peritoneal dialysis (CAPD) according to peritonitis risk, technique failure and patient survival in a large nation-wide PD cohort. This is a prospective cohort study that included all incident PD patients with at least 90 days of PD recruited in the BRAZPD study. All patients who were treated exclusively with either APD or CAPD were matched for 15 different covariates using a propensity score calculated with the nearest neighbor method. Clinical outcomes analyzed were overall mortality, technique failure and time to first peritonitis. For all analysis we also adjusted the curves for the presence of competing risks with the Fine and Gray analysis. After the matching procedure, 2,890 patients were included in the analysis (1,445 in each group). Baseline characteristics were similar for all covariates including: age, diabetes, BMI, Center-experience, coronary artery disease, cancer, literacy, hypertension, race, previous HD, gender, pre-dialysis care, family income, peripheral artery disease and year of starting PD. Mortality rate was higher in CAPD patients (SHR1.44 CI95%1.21-1.71) compared to APD, but no difference was observed for technique failure (SHR0.83 CI95%0.69-1.02) nor for time till the first peritonitis episode (SHR0.96 CI95%0.93-1.11). In the first large PD cohort study with groups balanced for several covariates using propensity score matching, PD modality was not associated with differences in neither time to first peritonitis nor in technique failure. Nevertheless, patient survival was significantly better in APD patients.